
United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 
Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
www.uspto.gov 



APPLICATION NO. 


FILING DATE 


FIRST NAMED INVENTOR 


ATTORNEY DOCKET NO. 


CONFIRMATION NO. 


10/674,312 


09/30/2003 


Vivck Jain 


JP920030093US1 


9424 



7590 

T. Rao Coca 
IBM Corporation 
Almaden Research Center 
650 Harry Road 
San Jose, CA 95120 



07/11/2007 



EXAMINER 



AHMED, AFFAF 



ART UNIT 



PAPER NUMBER 



3609 



MAIL DATE 



DELIVERY MODE 



07/11/2007 



PAPER 



Please find below and/or attached an Office communication concerning this application or proceeding. 



The time period for reply, if any, is set in the attached communication. 



PTOL-90A (Rev, 04/07) 



Office Action Summary 

* 


Application No. 

10/674,312 


Applicant(s) 

JAIN ETAL 


Examiner 

Affaf Ahmed 


Art Unit 

3609 


< 



- The MAILING DA TE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 
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Detailed Action 
Status of Claims 

1 . This action is in reply to the Application filled on 09/30/2003. 

2. Claims 1-28 were canceled on 09/30/2003 and claims 29-56 were amended on 09/30/2003. 

3. Claims 29-56 are currently pending and have been examined. 



Information Disclosure Statement 

4. The Information Disclosure Statement filed on 09/30/2003 has been considered. An initialed copy of the 
Form 1449 is enclosed herewith. 

■ 

Drawings 

r 

5. The drawings are objected to under 37 CFR 1.83(a) because: 

• They fail to show item 214 and 2 10 as described in the specification. 

• Item 302 (fig 3) does not disclose the option of "customer being selected from a library of shopper profile" 
in the specification. 

Any structural detail that is essential for a proper understanding of the disclosed invention should be shown 
in the drawing. MPEP § 608.02(d). Corrected drawing sheets in compliance with 37 CFR 1.121(d) are 
required in reply to the Office action to avoid abandonment of the application. Correction is required. 
Any amended replacement-drawing sheet should include all of the figures appearing on the immediate prior 
version of the sheet, even if only one figure is being amended. The figure or figure number of an amended 
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drawing should not be labeled as "amended." If a drawing figure is to be canceled, the appropriate figure 
must be removed from the replacement sheet, and where necessary, the remaining figures must be 
renumbered and appropriate changes made to the brief description of the several views of the drawings for 
consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining 
figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top 
margin as either "Replacement Sheet" or "New Sheet" pursuant to 37 CFR 1.121(d). If the changes are not 
accepted by the examiner, the applicant will be notified and informed of any required corrective action in 

* 

the next Office action. The objection to the drawings will not be held in abeyance. 



Specification 

6. The disclosure is objected to because item 214 (fig 2) referred to "Reinforcement learning in constrained 
domain tool", and item 210 (fig2) refereed to "Merchant objective specification tool", while the drawing 
shows item 214 refereed to "Merchant objective specification tool" and item 210 referred to 
"Reinforcement learning in constrained domain tool". Appropriate correction is required. 

7. The disclosure is objected to because item 302 (fig 3) is not specified in the specification. Appropriate 
correction is required. 

■ 

Claim Rejections - 35 USC § 112 

The following is a quotation of the first paragraph of 35 U.S.C. 1 12: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to 
which it pertains, or with which it is most nearly connected, to make and use the same and shall set forth 
the best mode contemplated by the inventor of carrying out his invention. 
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8. Claim 44 recites the limitation of "a first sub-unit operable for determining sequences in which selected 
initiatives can be deployed, if more than one initiative is selected." It has been held that actions that may 
or may not be done is indefinite and does not distinguish the claim from the prior art. (In re Collier, 158 
USPQ 266 (CCPA 1968)). 

9. Claim 50 recites the limitation of "determining sequences in which selected initiatives can be deployed, if 
more than one initiative is selected". It has been held that actions that may or may not be done is indefinite 
and does not distinguish the claim from the prior art. (In re Collier, 158 USPQ 266 (CCPA 1968)). 



Claim Rejections-35 USC § 102 

10. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the 

rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

((b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 



11. Claims 29, 30-34,37-38,40,43-45,47-49,50-51,54-56 are rejected under 35 U.S.C. 102 (e) as being 
anticipated by Honarvar et al, US 6,321,206 B2. 

» 

Examiner's Note: The Examiner has pointed out particular references contained in the prior art of record 
within the body of this action for the convenience of the Applicant. Although the specified citations are 

♦ * 

representative of the teachings in the art and are applied to the specific limitations within the individual 
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claim, other passages and figures . may apply. Applicant, in preparing the response, should consider fully 
the entire reference as potentially teaching all or part of the claimed invention, as well as the context of the 
passage as taught by the prior art or disclosed by the Examiner. 

Claims 29 43, and 49: 

With regard to the limitations of: 

• Generating a plurality of marketing strategies; 

• Determining an optimal marketing strategy based on a state of a customer and constraints corresponding 
to marketing channels; 

• Deploying the determined optimal marketing strategy; 

• Recording customer response to the deployed optimal marketing strategy; 

• Updating information corresponding to the state of a customer based on the recorded customer response; 

• Repeating steps b to e for the specified time period. 

Honarvar teaches a system that applies different marketing strategies to different customers with an input 
unit and data processing unit to evaluate data for optimizing strategies options that will best satisfies the 
customer outcomes. See at least column 6, lines 34-62. The software-based system further determines an 
appropriate action that has a specific collection procedure or a specific marketing action. See at least 
column 2, lines 63-67 Further more, Honvarvar also teaches step of collecting data and periodically 
aggregated and grouped into different categories where an observation point is taken. See at least column 
11, lines 49-54. 

* 

Claims 30, 44 and 50; 

With regard to the limitations of: 

• Selecting at least one initiative that enables an addressing of the specified objective. 
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• Determining sequences in which selected initiatives can be deployed, if more than one initiative is 
selected. 

• Combining the selected initiatives in the determined sequences to obtain the plurality of marketing 
strategies. 

See at least column 3, lines 1 1-3 land column 5, lines 14-25 and figure 7. 

Claims 31 and 32; 

With regard to the limitations of: 

• Varying parameters of initiatives to generate new initiatives. Honvarvar teaches a system that assigns 
clients to groups based on characteristics, where subsequently these groups will be separated based on 
different rules. See at least column 3, lines 20-32 and column 10, lines 13-20. 

• Varying deployment time of initiatives. Honarvar teaches an event that automatically generated due to 
client behavior or systematically produced at specified time intervals. See at least column 3, lines 1 1- 
20 and fig 11. 

Claims 33, 38, 45, 51 and 56: 
With regard to the limitations of: 

• Determining all possible states of customers. See at least column 3, lines 2 1 -24. 

• Determining an optimal policy for each state based on past data. See at least column 3, lines 25-28. The 
above limitation is implicitly describing that based on consumer history or past data; consumers are being 
categorized into segments where the best policy will be applied. 

• Identifying the state of a customer, the customer visiting a merchant or the customer being selected from a 
database of customer. See at least column 3, lines 32-34. The above limitation implicitly consumers are 
randomly chosen from a database. 

• Identifying the optimal policy for an identified customer state. See at least column 3, lines 39-43. 



* 
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• Modeling customer's preferences for marketing channels, cost and effectiveness of different marketing 
channels, and the specified budget as effective constraints. Honvarvar teaches in at least fig 7, customer's 
preference of marketing channels; Honvarvar also teaches a decision management system that will increase 
cost- effectiveness of marketing strategies; see at least column 3, lines 4-6. Honvarvar does not specifically 
specify budget as effective constraint, however the above limitation is inherently implemented in decision 
management system where marketing strategy and advertising mechanisms are implemented. 

• Determining an optimal feasible policy based on effective constraints corresponding to marketing 
channels; and determining the optimal marketing strategy from the optimal feasible policy. Honarvar in at 
least fig 7 and associated text teaches a method of comparison between two marketing strategies and based 
on the outcome result the optimal marketing channel is implemented. 

A 

Claim 34: 

With regard to the limitations of: 

• Identifying all relevant attributes of customers. See at least column 7, lines 65- 67 and column 8, line 1 . 

• Partitioning the customers into partitions based on identified attributes using a similarity measure based 
on a historic policy, actual rewards and transition probabilities from one data point to another, the 
partitions forming new states of the customers. See at least column 5, lines 15-25 and column 10, lines 29- 
39. 

Claim 37, 47 and 53: 

> 

With regard to the limitations of: 

• Selecting the marketing, strategy, which maximizes a value for the state over all marketing strategies for a 
given state; and repeating said step of selecting for each state. See at least column 9, lines 9-40 and fig 7, 
and fig 1 1 as well as associated text. 



Claims 40, 48 and 54: 
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With regard to the limitations of: 

• Identifying a resulting state of the customer. See at least column 13, lines 52-55. 

• Updating values of the state of the customer. See at least column 13, lines 61-64. 

• Updating an optimal policy. See at least column 14, lines 54-67. Honarvar implicitly teaches the above 
three limitation by grouping customers into different categories, where by measuring the performance of 
each report group across test group Honarvar update the state of the consumers and moving them into a 
new category where a new hybrid strategy is applied to determine the best performing path decision of each 
test group. 

Claim 55: 

With regard to the limitations of: 

• A database storing information regarding initiatives that can be offered to customer. See at least column 
15, lines 47-49. 

• Marketing channels available for executing the initiatives. See at least fig 7 and associated text. 

• Cost and effectiveness of the marketing channels Honvarvar implicitly teaches the above limitation when 
implemented the comparison of targeting consumers either by placing calls or sending an offer. See at least 
fig 7 and associated text. 

• States of customers. See at least column 9, lines 34-42. 

• Enabling a merchant to specify at least one objective for a specified time period. See at least column 12, 
lines 51-56. 

• Generating a plurality of marketing strategies based on the objective specified by the merchant, the 

* 

marketing strategies being a combination of initiatives; and a component operable for determining the 
optimal marketing strategy and at least one marketing channel based on a state of a customer and cost and 
effectiveness of marketing channels. See at least the abstract. Honarvar implicitly teaches the above 
limitation when implemented the comparison of targeting consumers either by placing calls or sending an 
offer. See at least fig 7 and associated text. Further more, the limitation above of determining consumer 
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state and cost effectiveness are inherently done in decision marketing analysis to enhance consumer 
relationship and implementing the optimal solution with the least amount of effort, cost and risk. 

■ 

Claim 56: 

With regard to the limitations of: 

• Generating a plurality of marketing strategies. See at least column 6, lines 36-37. 

• Determining all possible states of customers. See at least column 3, lines 21-24. 

• Determining an optimal policy for each state based on past data. See at least column 3, lines 25-28. The 
above limitation is implicitly describing that based on consumer history or past data; consumers are being 
categorized into segments where the best policy will be applied. 

• Identifying the state of a customer, the customer visiting a merchant or the customer being selected from a 
database of customers. See at least column 3, lines 32-34. The above limitation implicitly consumers are 
randomly chosen from a database. 

• Identifying the optimal policy for an identified customer state. See at least column 3, lines 39-43. 

• Modeling customer's preferences for marketing channels, cost and effectiveness of different marketing 
channels, and the specified budget as effective constraints. Honvarvar teaches in at least fig 7, customer's 
preference of marketing channels. Honvarvar also teaches a decision management system that will increase 
cost- effectiveness of marketing strategies. See at least column 3, lines 4-6. Honvarvar does not 
specifically specify budget as effective constraint, however the above limitation is inherently implemented 
in decision management system where marketing strategy and advertising mechanisms are implemented 

• Determining an optimal feasible policy based on the identified optimal policy and effective constraints 

* 

corresponding to marketing channels. 

• Determining an optimal marketing strategy from the optimal feasible policy. 

Honarvar in at least fig 7 and associated text teaches a method of comparison between two marketing 
strategies and based on the outcome result the optimal marketing channel is implemented. 

• Deploying the determined optimal marketing strategy. 
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• Recording customer response to the deployed marketing strategy. 

• Identifying a resulting state of the customer. 

• Updating values of the state of the customer. 

• Updating the optimal policy. 

• Repeating steps c to m for the specified time period. 

Honvarvar teaches a system that applies different marketing strategies to different customers with an input 
unit and data processing unit to evaluate data for optimizing strategies options that will best satisfies the 
customer outcomes. See at least column 6, lines 34-62. The software-based system further determines an 
appropriate action that has a specific collection procedure or a specific marketing action. See at least 
column 2, lines 63-67 Further more, Honvarvar also teaches step of collecting data and periodically 
aggregated and grouped into different categories where an observation point is taken. See at least column 
11, lines 49-54. 

4 

Claim Rejections - 35 USC § 103 

12. The following is a quotation of 35 U.S.C 103(a) which forms the basis for all obviousness rejections set 

forth in this Office action: 

A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art 
are such that the subject matter as a whole would have been obvious at the time the invention was made to 
a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be 
negatived by the manner in which the invention was made. 

13. Claims 35,36,39,40-42, 46 and 52 are rejected under 35 U.S.C. 103(a) as being unpatentable over Honarvar 
et al, US Pat. No. 6,321,206 Bl in view of Ulwick et al, US. Pat. No. 6,1 15,691. 



/ 
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Claim 36, 40 and 41: 

Honarvar does teach the limitation of: 

• Computing transition probabilities from a given state to another state for the policy. See at least column 6, 
lines 52-67 and column 9, lines 1-4. Honarvar does not specifically teach the limitations of: 

• Computing value of expected immediate reward for the policy in the state. 

• Computing discounted expected value of a resulting state for the policy. 

• Computing a sum of expected immediate reward and the discounted expected value. 

• Computing a sum of a new immediate reward. 

• A discounted value corresponding to the resulting stat. 

• Reduced by a value corresponding to an initial state of the customer. 

• Updating the values corresponding to the initial state of the customer by adding a fraction of the computed 
sum to a value of a previous state of the customer. 

Ulwick however, in at least column 9, lines 2-4 and column 7, lines 46-50 teaches a computer software 
based process program that is capable of predicting metrics for measuring parameter of desired outcomes; 
computing and solving matrix analysis to determine the best feasible solution for certain data. However, 
it would have been obvious to one of ordinary skill in the art at the time of the invention to modify 
Honarvar's creating strategies techniques with Ulwick's strategies evaluation and optimization to compute 
discount and sum of rewards to maintain consumers loyalty. 

• Propagating a change in the value of the state to all other states. Ulwick teaches in at least column 6, lines 

53-67 and column 9, lines 5- 17 that based on parameters and effectiveness of predictive metrics of each 
assigned value and weight will determine consumer satisfaction with overall outcome result. It would have 
been obvious to one of ordinary skill in the art at the time of the invention to combine Honarvar's creating 
strategies techniques with Ulwick's strategies evaluation and optimization to obtain an optimum solution 
over time. 

w 

Claims 35,46 and 52: 
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Honarvar does not specifically teach the following limitations. 

• Identifying a deterministic policy. Ulwick teaches the parameters of each mission of the software are 
identified in advance. See at least column 6, lines 6-13. It would have been obvious to one of ordinary 
skill in the art at the time of the invention to modify Honvarvar's creating strategies techniques with 
Ulwick's strategies evaluation and optimization to obtain and implement the best strategy to maintain 
consumers loyalty. 

• Initializing a value of all possible states for the policy. Ulwick teaches initialization of data is done in 
advance and loaded and stored into the computer. See at least column 6, lines 17-21. It would have been 
obvious to one of ordinary skill in the art at the time of the invention to modify Honarvar's creating 
strategies techniques with Ulwick's strategies evaluation and optimization so no corrupted data is 
produced. 

• Computing the value of a state for the policy. Ulwick teaches a computer based system and software 
package that computes predictive metrics on each desired outcome based on assigned values and weights. 
See at least column 6, lines 53- 67 and fig 15 with associated texts. It would have been obvious to one of 
ordinary skill in the art at the time of the invention to modify Honarvar's creating strategies techniques with 
Ulwick's strategies evaluation and optimization to obtain the most effective and efficient result of the 
strategy. 

• Repeating said step of computing for all possible states. Ulwick teaches contemplate establishment of a 
finite set of predictive metrics. See at least column 16, lines 17-30. It would have been obvious to one of 
ordinary skill in the art at the time of the invention to modify Honarvar's creating strategies techniques with 
Ulwick's strategies evaluation and optimization to insure validity of the outcome results. 

• Constructing a new improved policy. Ulwick teaches computing of data will describe the entire outcome 
that an optimal solution will achieve. See at least column 6, lines 23-36. It would have been obvious to 
one of ordinary skill in the art at the time of the invention to modify Honarvar's creating strategies 
techniques with Ulwick's strategies evaluation and optimization to obtain the most effective result of the 
strategy. 
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• Iteratively performing steps of computing, repeating, and constructing until the new improved policy 
remains unchanged for two subsequent iterations. Ulwick teaches contemplate establishment of a finite set 
of predictive metrics. See at least column 21, lines 48-56 and column 16, lines 41-46. It would have been 

* 

obvious to insure the suitability and the technique that implement is beneficial and will produce an efficient 
outcome results. 

i 

• Selecting the policy with maximum value for the state as the optimal policy for the given state. Ulwick 
teaches result of computed data displayed in prioritizing of importance with maximum and minimum value 
of each predictive metric set. See at least column 16, lines 40-53, and fig 19-a with the associated texts. It 
would have been obvious to one of ordinary skill in the art at the time of the invention to modify 
Honarvar's creating strategies techniques with Ulwick's strategies evaluation and optimization To obtain 
the most desired solution of the outcome. 

Claim 39: 

With regard to the limitations of: 

• Determining an optimal feasible policy based on effective constraints corresponding to marketing 
channels comprises, Honvarvar in at least fig 7 and associated text teaches a method of comparison 
between two marketing strategies and based on the outcome result the optimal marketing channel is 
implemented. Honvarvar doe not specifically teach the limitation of: 

• Mapping the optimal policy uniquely to a closest feasible optimal policy based on the effective constraints, 
if the effective constraints are not satisfied by the optimal policy. Ulwick however teaches in at least fig 18 
c and associated text where the closest estimation of the feasibility region is implemented. It would have 
been obvious to one of ordinary skill in the art at the time of the invention to combine Honarvar's creating 
strategies techniques with Ulwick's strategies evaluation and optimization to create the optimal solution 
that will deliver the most value given effort, risk and cost that consumer is willing to expend. 
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Conclusion 



Any inquiry concerning this communication or earlier communications from the examiner should 
be directed to Affaf Ahmed whose telephone number is 571-270-1835. The examiner can normally be 
reached on Monday- Friday 7:30 AM- 5:00 PM EST, ALT Fridays Off. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, James 
Reagan can be reached on. The fax phone number for the organization where this application or 
proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent Application 
Information Retrieval (PAIR) system. Status information for published applications may be obtained from 
either Private PAIR or Public PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 

« 

866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or 

access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Name : Affaf Ahmed 

Title : Patent Examiner 

Date : 7/9/2007 

Signature: 





